Non-asymptotic superlinear convergence of standard quasi-Newton methods

Abstract In this paper, we study and prove the non-asymptotic superlinear convergence rate of Broyden class quasi-Newton algorithms which includes Davidon–Fletcher–Powell (DFP) method Broyden–Fletcher–Goldfarb–Shanno (BFGS) method. The asymptotic these methods has been extensively studied in literature, but their explicit finite–time local is not fully investigated. provide a (non-asymptotic) analysis for under assumptions that objective function strongly convex, its gradient Lipschitz continuous, Hessian continuous at optimal solution. We show neighborhood solution, iterates generated by both DFP BFGS converge to solution $$(1/k)^{k/2}$$ ( 1 / k ) 2 , where k number iterations. also similar result holds case self-concordant. Numerical experiments on several datasets confirm our bounds. Our theoretical guarantee one first results methods.